Analyzing and Measuring the Latency of the Totem Multicast Protocols

Multicast group communication protocols provide a foundation on which distributed systems can be built. To determine whether these distributed systems can meet their real-time requirements, it is necessary to determine the message delivery latency of the underlying group communication systems. This research determines, by analysis and measurement, the probability density functions (pdfs) for the latency from origination to message delivery in the presence of message loss and token loss for the Totem protocols, which provide reliable totally ordered multicasting of messages across single and multiple local-area networks (LANs). Totem uses a logical token-passing ring on each LAN with gateways that forward messages selectively between LANs. The main idea of the analysis of the pdfs for the latency involves decomposing the latency into independent components and convolving the pdfs of those components. Comparing the performance of single-ring, two-ring and four-ring networks, it is shown that, with message ltering in the gateways, multiple-ring networks achieve lower mean latency, less variability, and shorter tails of the latency distribution than an equivalent single ring. The experimental measurements were based on a single LAN (Ethernet) and two LANs (Ethernets) of Sun workstations connected by a gateway. The clock skew between the workstations is eliminated from the measured latencies by equating the sample means of the actual latencies. The measurements of the pdfs for the latency are similar to those obtained from the analytical formulas, particularly for values of the latency with high probabilities. Limitations of the analytical model, and insight about communication and scheduling of the Solaris operating system, are also discussed.